BY  THE  COMPTROLLER  GENERAL 


®  rSARs*-  Defense  Department  Reports 
o  rfiat  Should  Provide  More  Information 


jfojhe  Congress . 


Important  information  which  would  be  useful 
to  the  Congress  and  to  top-level  Defense 
Department  managers  is  being  omitted  from 
DOD's  selected  acquisition  reports.  These 
reports  tell  where  a  major  weapons  program 
stands  in  relation  to  its  planned  cost,  sched¬ 
ule,  and  performance.  Incomplete,  misleading, 
or  inaccurate  reporting  of  status  could  affect 
congressional  and  DOD  decisions  about  weap¬ 
ons  programs. 

GAO  recommends  that  DOD  improve  the  in¬ 
formation  the  reports  provide  and  enforce  its 
instructions  cn  preparing  the  reports. 
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Single  copies  of  GAO  reports  are  available  free  of 
charge.  Requests  (except  by  Members  of  Congress) 
for  additional  quantities  should  be  accompanied  by  pay¬ 
ment  of  $1.00  per  copy.  (Do  not  send  cash). 

Requests  for  free  single  copies  should  be  sent  to: 

U.S.  General  Accounting  Office 
Distribution  Section,  Room  1518 
441  G  Street,  NW 
Washington,  DC  20548 

Requests  for  multiple  copies  should  be  sent  with  checks 
or  money  orders  to 

U.S.  General  Accounting  Office 
Distribution  Section 
P.O.  Box  1020 
Washington,  DC  20013 

Checks  or  money  orders  should  be  made  payable  to 
the  U.S.  General  Accounting  Office. 

To  expedite  placing  your  order,  call  (202)  275-6241 . 
When  ordering  by  phone  or  mail,  use  the  report  number 
and  date  iri  the  lower  right  corner  of  the  front  cover. 


GAO  reports  are  now  available  on  microfiche.  If  such 
copies  will  meet  your  needs,  be  sure  to  specify  that  you 
want  microfiche  copies. 
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To  the  president  of  the  Senate  and  the 
Speaker  of  the  House  of  Representatives 

This  report  points  out  ways  in  which  the  Department 
of  Defense's  selected  acquisition  reporting  system  could 
be  changed  to  present  more  completely  the  status  and 
progress  of  major  weapon  systems  and  their  expected 
operational  capabilities  and  limitations.  We  made  this 
review  at  the  request  of  the  Chairman,  Subcommittee  on 
priorities  and  Economy  in  Government,  Joint  Economic 
Committee.  At  his  request,  we  did  not  obtain  formal 
agency  comments  due  to  the  time  that  would  have  been 
required . 

We  are  sending  copies  of  this  report  to  the  Director, 
Office  of  Management  and  Budget,  and  the  Secretary  of 
Defense.  , 
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Comptroller  General 
of  the  United  States 
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COMPTROLLER  GENERAL' S  " SARs"  —DEFENSE  DEPARTMENT 

REPORT  TO  THE  CONGRESS  REPORTS  THAT  SHOULD  PROV1DI 

MORE  INFORMATION  TO  THE 
CONGRESS 


DIGEST 


•^Selected  acquisition  reports  (SARs)  have 
'become  the  key  recurring  summary  reports 
on  the  progress  of  the  Department  of 
Defense's  (DOD's)  most  costly  acquisition 
programs.  SARs  are  usually  prepared  for 
about  50  major  weapon  systems  and  are  used  by 
both  the  Congress  and  top-level  DOD  managers 
in  making  decisions  affecting  those  systems. 
However,  important  information  which  would  be 
useful  to  management  and  which  is  called  for 
by  DOD  instructions  is  not  being  reported. , 


The  Congress  uses  SARs,  along  with  research, 
development,  test,  and  evaluation  descriptive 
/  summaries;  procurement  justifications;  congres¬ 

sional  data  sheets;  and  testimony  by  dod 
officials,  in  making  funding  and  other  deci- 
\  sions  on  major  system  acquisitions.  Congres¬ 

sional  staffs  supplement  these  sources  of 
information  with  direct  personal  contacts  and 
information  requests. 

jY 

GAO  has  continually  worked  with  DOD  and  with 
congressional  committees  to  improve  SARs. 

GAO's  annual  reviews  of  individual  weapons 
systems  often  have  included  SAR-related 
matters. 


In  GAO’s  opinion,  SARs  should  provide  a  full 
and  objective  disclosure  of  the  status  of 
major  systems.  Tb£L_£o  1-1  wing  improvements 
would  make  SARs  more  useful  to' DOD  and  the 
Congress.'  ■.  1 1' 

— SARs  should  contain  (in  accordance  with 
current  instructions)  an  assessment  of  how 
well  the  system  is  expected  to  satisfy  its 
mission  and  should  identify  those  areas  in 
which  it  will  fall  short.  The  assessment 
statement  should  relate  to  how  well  the 
system  will  perform  its  mission  in  the 
expected  operational  environment  rather 
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than  just  to  performance  specifications 
listed  in  the  report.  (See  pp.  5  and  6.) 

-The  status  of  related  systems  and  key 
subsystems  should  be  shown  (in  accordance 
with  current  instructions).  (See  pp.  9  and 
10.) 

-The  planning  estimate  should  be  included 
in  the  first  report  containing  the 
development  estimate  with  an  explanation 
for  changes  (in  accordance  with  current 
instructions)  and  should  be  retained 
on  subsequent  SARs.  (See  pp.  10  and  11.) 

-The  planning  and  development  cost  estimates 
should  be  stated  in  ranges  of  costs  rather 
than  specific  point  estimates.  (See  p.  13.) 

-Changes  to  development  estimates  should 
be  fully  explained  in  the  report  containing 
the  change ,  'ind  subsequent  SARb  should  pro¬ 
vide  a  reference  to  the  original  devel¬ 
opment  estimate.  (See  pp.  13  and  14.) 

-SARs  should  reveal  operational  and  technical 
risks  and  critical  test  issues.  (See  p.  15.) 

-Greater  consideration  should  be  given  to 
selected  acquisition  reporting  for  impor¬ 
tant  major  systems  still  in  the  early  stages 
of  advanced  development.  (See  pp.  15  to  17.) 

-Earlier  consideration  should  be  given  to 
deleting  from  selected  acquisition  reporting 
those  systems  that  are  near  the  end  of  pro¬ 
duction.  (See  pp.  16  and  17.) 

-SARs  should  include  additional  costs  re¬ 
quired  to  deploy  a  weapon  system  (logistic 
support/additional  procurement  costs). 

(See  pp.  17  to  20.) 

-SARs  should  include,  as  they  did  prior  to 
June  30,  1979,  a  chart  showing  the  effect 
of  using  different  escalation  rates  to 
estimate  program  acquisition  cost.  (See 
p.  20.) 


ii 


--The  Secretary  of  Defense,  or  a  designated 
official,  should  certify  as  to  the  credi¬ 
bility  of  the  reports.  (See  p.  22.) 

-- A  periodic  independent  review  should  be 
made  of  the  accuracy  and  completeness  of 
the  reports.  (See  pp.  20  to  22.) 

RECOMMENDATIONS 


The  Secretary  of  Defense  should  (1)  incorpo¬ 
rate  into  DOD  Instruction  7000.3  those  im¬ 
provements  presented  above  which  are  rot 
currently  included  and  (2)  enforce  the 
Instruction's  provisions. 

GAO  recognizes  that  it  is  difficult  for  DOD 
to  decide  what  information  to  include  in  SARs 
and  in  what  detail.  The  SARs  must  be  short 
enough  to  be  usable  by  people  who  have  little 
time  to  review  them  and  yet  SARs  should  present 
complete,  accurate  data  which  is  not  misleading. 

The  additional  data  GAO  is  recommending  that 
DOD  include  (such  as  operational  and  technical 
risks,  operational  capability  shortfalls, 
additional  costs  needed  to  deploy  a  system, 
and  planning  estimates)  is  the  kind  of  data 
it  may  not  want  to  include  because  it  detracts 
from  an  optimistic  presentation  of  system  capa¬ 
bilities  and  program  progress  and  status.  How¬ 
ever,  it  seems  to  be  the  kind  of  data  the 
Congress  needs  to  have  in  reviewing  and  funding 
programs.  (See  p.  23.) 
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INTRODUCTION 

The  Congress  authorizes  billions  of  dollars  annually  to 
buy  weapon  systems.  In  order  for  the  Congress  to  make  deci¬ 
sions  and  to  allocate  national  resources  among  Government 
programs,  it  is  essential  that  accurate  and  informative 
data  be  provided  on  the  status  and  progress  of  major  weapon 
systems  being  developed  and  deployed  by  the  Department  of 
Defense  (DOD). 

DOD  Instruction  7000.3  of  February  23,  1968,  established 
the  selected  acquisition  report  (SAR)  requirement.  The 
SAR's  initial  purpose  was  to  keep  its  sponsor,  the  Assistant 
Secretary  of  Defense  (Comptroller),  apprised  of  the  progress 
of  selected  acquisitions  and  to  compare  this  progress  with 
planned  technical,  schedule,  and  cost  performance. 

In  February  1969  the  Chairman  of  the  Senate  Armed 
Services  Committee  asked  the  Secretary  of  Defense  to  provide 
status  reports  on  major  weapon  systems.  The  parties  agreed 
in  April  1969  that  the  SAR  would  be  the  vehicle  to  satisfy 
the  committee's  needs.  As  a  result,  the  SAR  became  and 
remains  the  key  recurring  summary  report  on  the  progress 
of  selected  major  acquisition  programs. 

SARs  are  usually  prepared  on  about  50  major  weapon 
systems.  Because  SAR  coverage  normally  begins  after  a  system 
enters  full-scale  development,  many  major  systems  in  advanced 
development  are  excluded.  In  addition,  most  modification 
projects  are  excluded  from  the  reporting,  even  though 
many  exceed  the  major  weapon  system  criteria. 

INTEREST  IN  SAR  IMPROVEMENTS 

The  Senate  and  House  Armed  Services  and  Appropriations 
Committees  are  the  primary  congressional  users  of  SARs.  These 
committees  have  long  been  concerned  with  acquiring  adequate 
information  on  the  progress  of  major  weapon  systems,  particu¬ 
larly  those  in  the  early  phases  of  the  acquisition  process  when 
numerous  options  are  available  to  the  Congress.  Other  im¬ 
portant  congressional  committees  that  use  SARs  are  the  House 
and  Senate  Budget  Committees  and  the  House  Government 
Operations  Committee. 

In  1975  the  Congress  enacted  Public  Law  94-106  estab¬ 
lishing  a  legal  reporting  requirement  for  SARs.  Section 
811  provided  that: 
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"(a)  Beginning  with  the  quarter  ending  December  31 , 

1975,  the  Secretary  of  Defense  shall  submit  to  the 
Congress  within  30  days  after  the  end  of  each  quar¬ 
ter  of  each  fiscal  year,  written  selected  acquisi¬ 
tion  reports  for  those  major  defense  systems  which 
are  estimated  to  require  the  total  cumulative  financ¬ 
ing  for  research,  development,  test,  and  evaluation 
in  excess  of  $50,000,000  or  a  cumulative  production 
investment  in  excess  of  $200,000,000.  If  the 
reports  received  are  preliminary  then  final  reports 
are  to  be  submitted  to  the  Congress  within  45  days 
after  the  end  of  each  quarter. 

"(b)  Any  report  required  to  be  submitted  under  sub¬ 
section  (a)  shall  include,  but  not  be  limited  to, 
the  detailed  and  summarized  information  included 
in  reports  required  by  section  139  of  title  10, 

United  States  Code."  1/ 

Since  public  Law  94-106  was  enacted,  the  dollar  thresh¬ 
olds  defining  a  major  weapon  system  have  been  raised  to 
$75  million  for  research,  development,  test,  and  evaluation 
and  $300  million  for  production.  In  addition,  DOD's  Authori¬ 
zation  Act  for  fiscal  year  1980  modified  the  reporting  pro¬ 
cedures  to  require  that  (1)  reports  for  quarters  ending  on 
December  31  be  submitted  within  20  days  after  the  President 
transmits  the  budget  to  the  Congress  for  the  following  year 
and  (2)  the  final  report  for  any  quarter  in  which  a  prelimi¬ 
nary  report  is  submitted  to  the  Congress  be  submitted  within 
15  days  after  submission  of  the  preliminary  report. 

The  above  congressional  actions,  as  well  as  (1)  the  re¬ 
quest  which  resulted  in  our  review  and  (2)  June  25,  1979, 
hearings  before  the  Legislation  and  National  Security 
Subcommittee,  House  Committee  on  Government  Operations,  on 
"Inaccuracy  of  Department  of  Defense  Weapons  Acquisition 
Cost  Estimates,"  clearly  demonstrate  the  congressional 
interest  in  the  quality  and  completeness  of  SAR  data. 


1/Section  139  of  Title  10,  U.  S.  Code,  requires  DOD  to 
~  report  operational  test  results  for  major  weapon  systems 
for  which  procurement  funds  are  reauested.  These  test 
results  and  other  information  are  included  in  congres¬ 
sional  data  sheets  submitted  annually  to  the  Congress  with 
the  President's  budget. 
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SCOPE  AND  METHODOLOGY 


To  comply  with  the  request  that  we  review  the  selected 
acquisition  reporting  system  to  evaluate  its  adequacy  and 
to  suggest  improvements  in  the  system,  we: 

— Interviewed  officials  at  system  program  offices, 
intermediate  and  higher  commands  of  the  military 
departments,  and  the  Office  of  the  Secretary  of 
Defense  to  determine  (1)  their  positions  on  present 
reporting  policy  and  report  processing  practices, 

(2)  acceptance  of  our  prior  recommendations,  and 

(3)  reasons  for  noncompliance  with  DOD  instructions. 

— Compared  underlying  system  cost,  schedule,  and 
performance  data  to  that  being  reported. 

We  directed  our  attention  primarily  to  the  program 
highlights,  operational/technical  characteristics, 
schedule  milestones,  and  program  acquisition  cost  sections 
of  the  reports.  We  reviewed  all  SARs  of  various  dates  for 
some  parts  of  our  review.  For  other  parts,  we  reviewed 
SARs  for  up  to  27  systems  which  were  being  reviewed  or  had 
been  reviewed  in  1978-79  as  part  of  our  annual  reviews  of 
major  weapon  systems.  We  selected  these  systems  to  make 
maximum  use  of  available  data. 

As  requested,  we  did  not  obtain  formal  agency  comments 
on  the  report.  The  issues  identified,  however,  were  discussed 
with  agency  officials.  We  have  omitted  all  classified  data 
in  order  to  issue  an  unclassified  report. 
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CHAPTER  2 


BETTER  SARs  COULD  IMPROVE 
MANAGEMENT  OF  MAJOR  WEAPON  SYSTEMS 

Both  the  Congress  and  DOD’s  top-level  management  rely  on 
SARs  to  tell  them  where  a  program  stands  relative  to  its 
planned  cost,  schedule,  and  performance.  Incomplete,  mislead¬ 
ing,  or  inaccurate  reporting  of  status  could  result  in  con¬ 
gressional  and  DOD  decisions  that  would  not  otherwise  be  made. 
We  found  that  important  data  wnich,  in  our  opinion,  should  be 
available  for  use  in  the  decisionmaking  process  is  not  being 
prov ided : 

— A  required  assessment  of  how  well  the  system  is 
expected  to  satisfy  its  mission  requirement,  in¬ 
cluding  an  identification  of  those  areas  in  which  it 
will  fall  short,  is  not  always  being  provided. 

— Status  information  which  is  required  on  subsystems 
and  related  systems  is  not  always  being  provided. 

— Systems'  planning  estimates  are  almost  never  re¬ 
ported  even  though  they  are  required  by  DOD's 
reporting  instructions. 

— Changes  to  a  program's  development  estimate  have 
not  always  been  fully  explained,  and  most  SARs  that 
have  had  changes  do  not  refer  to  the  original  de¬ 
velopment  estimate. 

— SARs  do  not  reveal  data  on  operational  and 
technical  risks  or  critical  test  issues. 

— Important  weapon  systems  involving  significant 
advanced  development  funding  are  almost  always 
excluded  from  SAR  coverage  until  after  the  full- 
scale  development  decision. 

— Some  costs  of  deploying  a  system  (logistic 

support  and  additional  procurement  costs)  are  nor¬ 
mally  excluded  from  SARs. 

--The  effect  of  using  different  escalation  rates  is 
no  longer  being  shown. 

In  addition,  some  systems  in  the  selected  acquisition  re¬ 
porting  system  do  not  appear  to  warrant  continued  reporting. 
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We  also  found  that  there  is  no  periodic,  independent 
review  of  the  accuracy  and  completeness  of  SAR  data  being  pro 
vided  and  that  no  one  certifies  to  the  credibility  of  SARs. 


ASSESSING  HOW  WELL  A  SYSTEM  WILL 
SATISFY  ITS  MISSION  REQUIREMENTS 

In  our  1975  report  we  recommended  that  each  SAR  include 
an  assessment  as  to  how  well  the  system  is  expected  to 
satisfy  its  mission  requirements.  This  improvement  was  incor 
porated  into  the  SAR  Instruction  in  September  1975  as  follows 

"Program  Highlights.  *  *  *  This  section  should 
also  provide  an  objective  assessment  as  to  the 
extent  to  which  the  system  is  expected  to  sat¬ 
isfy  its  current  mission  requirements,  identify¬ 
ing  those  areas  where  it  will  fall  short  of  such 
objectives. " 

This  data,  in  our  opinion,  is  among  the  most  subjective 
and  difficult  that  program  managers  are  to  provide  in  the  SAF 
On  the  other  hand,  it  is  among  the  most  important  and  useful. 

Assessment  is  not  being 
reported  on  some  SARs 

Although  the  mission  assessment  statement  has  been  a 
requirement  since  1975,  the  Office  of  the  Secretary  of 
Defense  and  the  military  services  have  been  very  lax  in 
requiring  program  managers  to  provide  them.  Our  review 
of  the  SARs  at  December  31,  1978,  showed  that  25  of  52 
SARs  which  DOD  sent  to  the  Congress  did  not  include  a  mis¬ 
sion  assessment  statement,  1/  and  only  1  Army  SAR  (Roland) 
identified  any  shot tcoming s' in  the  system's  ability  to 
accomplish  its  mission  requirements. 

During  our  review  officials  in  the  Office  of  the 
Assistant  Secretary  of  Defense  (Comptroller)  said  they 
had  overlooked  the  requirement  for  the  mission  assessment, 
but  action  would  be  taken  to  see  that  future  SARs  include 
this  statement.  Our  subsequent  review  of  the  September  3C , 
1979,  SARs  showed  that  12  SARs  sent  to  the  Congress  siill 
did  not  include  this  statement.  These  SARs  were  for  the 
Petshinj  II,  Hellfire,  Stand-Off  Target  Acquisition 
System,  Division  Air  Defense  Gun,  M-198  Howitzer,  Captor, 
Harpoon,  AIM-9L  Sidewinder,  MK-48  Torpedo  (Mod.  1), 


1/Of  five  additional  SARs,  which  were  not  sent  to  the 
Congress,  two  did  not  contain  an  assessment  statement. 


....  . . to  f'-f  k’’ 


SSN-688  Submarine,  General  Purpose  Amphibious  Assault  Ship 
(LHA),  and  CVM-Class  Carrier. 


The  assessment  statement 
should  be  expanded 

Congressional  hearings  and  our  studies  of  individual 
weapon  systems  reveal  numerous  limitations  and  shortcomings 
of  systems  in  their  expected  operational  environment.  Because 
of  the  narrow  definition  applied  to  the  assessment  statement, 
however,  these  factors  are  often  excluded  from  the  SARs .  We 
believe  the  SAR  would  be  a  more  useful  report  if  it  summarized 
these  shortcomings  and  limitations. 


The  assessment  statement,  when  included,  is  normally 
directed  toward  performance  characteristics  listed  in  the 
report  or  in  a  Decision  Coordinating  Paper  (DCP)  1/  and  often 
does  not  fully  convey  how  well  the  system  will  perform  in  its 
expected  operational  environment.  Moreover,  the  statement 
can  be  misinterpreted  since  it  follows  a  section  entitled 
"Mission  and  Description,"  which  describes  the  planned 
operational  use  of  the  system.  Although  the  assessment 
statement  normally  does  not  address  this  mission  description, 
it  could  be  interpreted  as  such  an  assessment  because  it  is 
located  so  close  to  the  mission  description.  In  our  opinion, 
therefore,  some  SARs  do  not  give  the  Congress  a  good  pic¬ 
ture  of  how  well  the  systems  are  expected  to  perform,  espe¬ 
cially  regarding  system  limitations. 


SARs  may  not  reflect  available  information  within  DOD  on 
expected  system  limitations  unless  these  limitations  are  di¬ 
rectly  related  to  factors  in  the  operational  and  technical 
characteristics  section.  Moreover,  even  these  limitations 
may  not  be  shown  because  the  SARs  current  estimate  seems  to 
reflect  the  optimistic  attitude  that  approved  program  goals 
will  be  met  as  long  as  efforts  are  underway  to  solve  specific 
problems.  For  example,  testing  and  other  data  indicate 
problems  with  (1)  the  XMl's  maintainability  and  reliability, 
(2)  Copperhead's  ability  to  meet  its  effectiveness  and 
single-shot  kill  probability  goals,  and  (3)  the  capability 
of  the  High  Speed  Anti-Radiation  Missile  (HARM).  Yet,  SARs 
give  little  or  no  indication  of  these  problems. 

The  following  ere  examples  of  systems  which  do  not  fully 
present  their  expected  operational  capabilities  and 
1  imitations . 


^/Performance  specifications  in  the  SARs  are  generally 
selected  performance  specifications  listed  in  a  DCP. 
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Copperhead 


The  September  30,  1979,  Copperhead  SAR  states:  "It  is 
expected  that  the  Copperhead  will  satisfy  its  current 
mission  requirement."  The  Copperhead  mission  is  described  as: 

"This  projectile  wi'l  be  employed  in  indirect  fire 
by  155mm  units  to  destroy  or  neutralize  moving 
and  stationary  hardpoint  targets  such  as  armored 
and  mechanized  vehicles  and  field  fortifications." 

In  addition,  the  SAR  includes  a  single-shot  kill  probability 
which  is  a  function  of  reliability,  hit  proDability, 
lethality,  probability  of  proper  launch,  and  probability  of 
proper  laser  designation  (illumination)  of  the  target. 

In  our  opinion,  the  Copperhead's  capability  as  presented 
in  the  SAR  is  misleading.  The  SAR  does  not  state  that  the 
single-shot  kill  probability  of  Copperhead  is  based  on  its 
expected  lethality  against  a  specific  target.  Its  lethality 
against  some  other  targets  is  expected  to  be  significantly 
less.  In  addition,  the  projectile  effectiveness  goal  does  not 
fully  consider  environmental  and  operational  factors,  such  as 
adverse  weather,  battlefield  conditions,  and  enemy  countermea¬ 
sures,  which  can  defeat,  degrade,  or  deny  the  use  of  Copperhead, 
primarily  by  limiting  or  preventing  the  required  visibility  be¬ 
tween  the  target  and  either  the  projectile  or  the  laser  desig¬ 
nator.  The  September  30,  1979,  SAR  also  does  not  present  re¬ 
cently  completed  operational  testing  which  resulted  in  less 
favorable  projectile  performance  than  called  for  in  the  SAR' s 
program  goal  and  current  estimate.  This  performance  resulted 
even  though  test  conditions  were  more  favorable  than  expected 
operational  conditions. 

Roland 

The  Roland  air  defense  weapon  system  is  described  as  an 
all-weather  system  which,  for  example,  is  to  operate  in  a 
specif'?  level  of  rain  per  hour.  Although  all-weather 
capability  is  a  prime  consideration,  the  Roland  SAR  contains 
no  operational  or  technical  characteristics  which  relate 
to  this  requirement.  The  September  30,  1979,  SAR  assess¬ 
ment  of  system  capability  states  that: 

"Based  on  test  results/analysis  to  date,  system 
reliability  is  less  than  required.  A  reliability 
improvement  program  is  underway  and  it  is  estimated 
that  all  approved  system  operational/technical 
requirements  will  be  achieved." 
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We  believe  that  the  SAR  is  misleading  since  it  does  not 
acknowledge  factors  which  indicate  that  the  desired 
operational  effectiveness  may  not  be  achieved.  These 
factors  include: 

— Modifications  required  to  meet  certain  aspects 
of  the  threat. 

— Problems  with  performance  in  rain.  (A  modifica¬ 
tion  has  been  proposed  to  improve  performance, 
and  additional  testing  is  planned.) 

— A  conclusion  by  the  Army  test  agency  that  Roland 
effectiveness  should  be  further  examined  with 
emphasis  on  performance  relative  to  a  specific 
aspect  of  the  threat  and  on  system  reliability 
and  maintainability. 

— Poor  ability  to  identify  "friend  or  foe"  during  op¬ 
erational  testing.  (Improvements  are  planned.) 

— Uncertainty  concerning  the  system's  ability  to  meet 
a  requirement  for  a  72-hour  mission  using  only 
organizational  maintenance. 

Precision  Location  Strike  System 

The  Precision  Location  Strike  System  (PLSS)  SAR  does  not 
present  some  pertinent  performance  characteristics  which 
would  provide  the  Congress  useful  information.  The  SAR 
shows  the  estimated  location  accuracy  for  finding  pulsed 
emitters  and  the  estimated  accuracy  for  guided  weapons. 

It  does  not  show  PLSS's  location  accuracy  for  finding 
enemy  radar  jammers  or  its  strike  accuracy  with  unguiaed 
weapons.  These  characteristics  are  pertinent  in  assessing 
PLSS's  ability  to  meet  its  mission. 

In  addition,  the  reported  strike  accuracy  is  ambiguous 
since  it  could  be  interpreted  as  the  distance  within  which 
a  veapon  should  hit  either  the  target  or  the  point  desig¬ 
nated  as  the  target  location. 


’*s  5-Inch,  Laser-Guided 


Project  lie 


The  Navy's  SAR  for  the  5-Inch,  Laser-Guided  Projectile 
contains  approved  program  goals,  demonstrated  performance, 
and  current  estimates  for  numerous  operational  and  technical 
characteristics.  The  characteristics  listed  are  for 
accuracy,  range,  reliability,  lethality,  weight,  length, 
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and  seeker  sensitivity.  These  factors  do  not  present  how 
effective  the  projectile  will  be  against  its  intended  tar¬ 
gets  in  the  expected  operational  environment.  Additional 
data,  such  as  single-shot  probability  of  kill  against  the 
intended  targets  and  qualifications  regarding  environmental 
and  operational  factors,  would,  in  our  opinion,  make  the 
SAR  more  meaningful  to  the  Congress. 


F-.14A  and  phoenix 

The  F-14A  and  Phoenix  SARs  describe  a  c 
intercept  six  targets  simultaneously  and  to 
nearly  simultaneously  against  six  targets, 
that  this  is  a  proven  operational  capability 
however,  the  F-14A  and  Phoenix  demonstrated- 
rathcr  than  operational  testing--the  capabil 
and  launch  against  six  target  drones  in  a  co 
environment.  Actual  launch  was  not  demonstr 
capability  to  intercept  and  launch  against  s 
a  realistic  operational  environment  has  not 
tr ated  . 


apebility  to 
launch  missiles 
The  SARs  imply 
.  In  fact, 

-in  development 
ity  to  track 
ntrolled 
ated.  The 
ix  targets  in 
been  demons- 


In  addition,  the  F-14A  SAR  describes  a  capability  for 
an  air-to-sur f ace  attack  mission.  This  data  is  misleading. 
The  Navy's  independent  test  agency  has  questioned  the  F-14A’s 
effectiveness  in  this  role. 

INCLUDING  DATA  ON  THE  STATUS  OF 


RELATED  SYSTEMS  AND  KEY  SUBSYSTEMS 


As  a  result  of  our  March  1975  recommendation  that  the 
status  of  related  systems  and  key  subsystems  be  shown  in 
SARs,  DOD  revised  the  SAR  instruction  as  follows: 

"Program  Highlights.  Briefly  summarize  the  signif¬ 
icant  developments  in  the  program,  including  the 
current  stilus  of  related  systems  and  key  subsys¬ 
tems,  except  for  those  covered  by  separate  SAR's." 

According  to  DOD  officials,  this  requirement  normally 
calls  for  only  a  brief  comment  on  the  status  of  key  subsystems 
and  related  systems.  Exceptions  are  that  when  the  status  of 
a  related  system  affects  the  performance  of  the  major  system 
or  when  the  status  of  a  key  subsystem  adversely  affects  the 
cost,  schedule,  or  performance  of  the  major  system,  it  should 
be  fully  reported.  We  agree  with  these  statements.  We  found, 
however,  that  SARs  sometimes  present  little  or  no  data  on  key 
subsystems  or  related  systems.  As  a  result,  SARs  sometimes 
present  the  status  of  a  particular  development  effort  but  not 
a  total  weapon  system. 
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For  example,  the  GBU-15  weapon  system  is  related  to 
PLSS.  The  Air  Force  has  planned  to  use  this  weapon  with  PLSS 
to  meet  its  mission  requirements.  There  is,  however,  a  serious 
question  about  the  GBU-15' s  ability  to  perform  effectively. 

The  PLSS  SAR  does  not  describe  the  status  of  this  related  sys¬ 
tem.  An  Air  Force  official  involved  in  SAR  processing  said 
that  it  would  not  be  in  the  best  interest  of  the  PLSS  program 
to  air  GBU-15  problems  in  its  SAR. 

Another  example  is  the  Army's  Copperhead  program.  This 
laser-guided,  artillery-fired  projectile  requires  that  a  laser 
designator  illuminate  the  intended  target  with  laser  energy. 

The  primary  laser  designator  is  to  be  the  Ground  Laser  Locator 
Designator.  Use  of  airborne  laser  designators  which  will 
operate  from  remotely  piloted  vehicles  and  from  helicopters 
is  being  considered.  However,  the  Copperhead  SAR  does  not 
present  the  status  of  these  laser  designators.  It  mentions 
the  possible  airborne  designators,  identifies  the  Ground 
Laser  Locator  Designator,  and  presents  a  performance 
characteristic  entitled  "Operational  Probability  of  proper 
Designation."  Reporting  the  status  of  related  systems  is 
especially  needed  for  Copperhead  oecause  the  Ground  Laser 
Locator  Designator  and  remotely  piloted  vehicles  are 
not  covered  in  other  SARs  and  because  in  late  1977  the 
designator  underwent  an  operational  test,  prior  to  entering 
production,  in  which  numerous  operational  problems  and 
concerns  were  identified. 

Less  than  25  percent  of  the  SARs  we  reviewed  mentioned 
the  status  of  subsystems.  Of  the  SARs  which  do,  the  Decem¬ 
ber  31,  1978,  SSN-688  SAR  was  a  good  example.  It  provided  the 
current  status  of  seven  key  subsystems. 

In  contrast,  SARs  for  the  Air  and  Ground  Launched  Cruise 
Missiles  and  Tomahawk  Cruise  Missile  did  not  provide  infor¬ 
mation  on  the  status  of  key  subsystems.  The  success  of  these 
missiles  depends  on  the  ability  of  the  lightweight  turbofan 
engine  to  provide  long-range  capability  and  of  the  Terrain 
Contour  Matching  System  to  periodically  update  an  inertial 
guidance  system.  The  Terrain  Contour  Matching  System 
is  critical  to  achieving  the  accuracy  that  cruise  missiles 
require  to  accomplish  their  mission  requirements.  The  current 
cruise  missile  SARs  list  subsystems  and  their  contractors  out 
do  not  comment  on  their  status. 

REPORTING  PLANNING  ESTIMATES 

The  planning  estimate  is  the  estimate  of  operational/ 
technical  characteristics,  schedule,  and  cost  developed  at 
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the  time  the  Secretary  of  Defense  approves  program  initia¬ 
tion — the  start  of  advanced  development.  It  is  also  the 
first  estimate  used  to  gain  congressional  approval.  We 
have  previously  recommended  that  planning  estimates  be 
included  on  SARs  to  provide  greater  visibility  over  programs 
as  they  progress. 

SAR  instructions  since  1975  have  required  that  the 
initial  SAR  which  shows  the  development  estimate  for  a 
weapon  system  shall  include  the  planning  estimate  and  an 
analysis  of  the  variance  between  the  planning  and  development 
estimates.  Instructions  also  require  that  a  copy  of  this 
analysis  be  attached  to  each  subsequent  SAR.  DOD  officials 
advised,  however,  that  the  planning  estimate  will  be  shown 
on  the  SAR  only  in  those  instances  in  which  a  system  has  not 
reached  Milestone  II  (approval  for  full-scale  engineering 
development)  and  a  development  estimate  is  not  yet  available. 
Since  systems  are  normally  added  to  SAR  after  they  pass 
Milestone  II,  most  systems  have  a  development  estimate  for 
the  initial  SAR;  thus,  the  planning  estimate  is  not  reported. 

Only  1  of  the  last  15  systems  added  to  the  reporting 
system  has  had  its  planning  estimate  reported.  The  one 
exception  was  the  Army's  Multiple  Launch  Rocket  System,  1/ 
which  was  added  in  June  1979.  Interest  in  this  system  dic¬ 
tated  that  it  be  given  SAR  coverage  before  Milestone  II,  and 
thus  the  initial  SAR  contained  its  planning  estimate.  Ear¬ 
lier,  the  XM1  tank  and  the  Advanced  Attack  Helicopter  ( AAH) 
were  also  added  to  the  SAR  before  reaching  Milestone  II 
because  of  congressional  interest.  These  SARs  included 
planning  estimates  until  the  systems  reached  Milestone  II  and 
the  development  estimate  was  reported.  The  required  variance 
analysis  was  included,  and  in  subsequent  SARs  the  planning 
estimate  was  dropped  in  accordance  with  DOD's  instructions. 
However,  a  copy  of  the  variance  analysis  was  not  attached  to 
subsequent  SARs  as  required  by  the  instructions. 

The  value  of  reporting  planning  estimates  is  illustrated 
by  the  cost  estimate  changes  which  occurred  between  program 
initiation  and  full-scale  development  for  the  XM1  and  AAH 
systems : 


1/Formerly  the  General  Support  Rocket  System. 
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Planning  Development 

System  estimate  estimate  Increase 

- (millions) - 

XM1  $3,005  $4,779  $1,774 

AAH  1,800  3,758  1,958 

Schedule  and  performance  changes  reflected  in  the  develop¬ 
ment  estimates  for  these  systems  included 

—  initial  operational  capability  for  the  AAH  over 
2-1/2  years  later  than  in  the  planning  estimate, 

— related  schedule  changes  for  other  AAH  milestones, 

— a  reduction  of  over  50  percent  in  certain  expendable 
ordnance  on  the  AAH, 

— reduction  in  the  higher  values  of  reliability  and 
ranges  for  the  XMl ,  and 

— schedule  changes  of  from  to  6  months  for 
the  XMl. 

We  believe,  therefore,  that  '  important  that  the  initial 

SAR  showing  a  development  estimate  explain  reasons  for 
changes  between  it  and  the  planning  estimate.  We  also  be¬ 
lieve  that  the  planning  estimate  should  be  shown  in  subse¬ 
quent  SARs  to  provide  greater  visibility.  This  information 
would  provide  more  complete  visibility  over  programs  and  would 
provide  valuable  insight  into  the  nature  of  planning  esti¬ 
mates  and  the  confidence  that  can  be  placed  on  them  for  new 
programs . 

DOD  officials  disagreed  with  our  position  on  reporting 
planning  estimates.  They  said  that  planning  estimates  are 
shown  when  (1)  they  can  be  defined  during  the  advanced  devel¬ 
opment  phase  and  (2)  a  request  has  been  made  by  one  of  the 
congressional  oversight  committees  for  reporting  before 
Defense  Systems  Acquisition  Review  Council  II.  They  stated, 
however,  that: 

— Planning  estimates  are  oriented  to  advanced  devel¬ 
opment  in  which  candidate  hardware  components  of 
the  ultimate  major  system  are  demonstrated  and 
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validated  to  determine  whether  a  weapon  system  can 
be  developed.  As  such,  planning  estimates  usually 
cannot  be  quantified  in  sufficient  detail  to  serve 
as  a  valid  baseline. 

— The  planning  estimates  usually  cannot  be  quantified 
in  sufficient  detail  to  permit  the  initiation  of 
SAR  reporting. 

— The  program  reflected  in  the  planning  estimates 
is  often  not  comparable  to  the  program  reflected 
in  the  development  estimate,  often  because  planning 
estimates  contain  data  on  more  than  one  alternative. 

We  recognize  that  in  instances  when  a  Defense  Systems 
Acquisition  Review  Council  is  not  held  prior  to  initiation 
of  a  program,  the  cost,  schedule,  and  performance  estimates 
that  are  available  may  not  be  well  defined.  We  believe, 
however,  that  SARs  should  include  planning  estimates  in 
whatever  level  of  definition  exists  with  a  brief  explanation 
for  the  differences  between  the  planning  estimates  and 
the  development  estimates,  when  the  latter  are  first  re¬ 
ported,  so  that  more  complete  visibility  over  a  program  is 
possible . 

£  nee  planning  estimates  cannot  be  well  d  *ed  relative 
to  the  final  progam  and  since  planning  and  de  nent  cost 

estii.it  es  ^ave  normal1 y  proven  to  be  signific.  i  less 
than  acv.al  program  tjst:  we  also  believe  that  these  cost 
estimates  should  be  states  in  ranges  of  costs  rather  than 
specific  point  estimates.  Tb ' s  change  also  was  recently 
recommended  by  the  House  Committee  on  Government  Operations 
(H.R.  96-656)  . 

REPORT  ,NG  CHANGES  TO 
DEVELQPMF,.  T  ESTIMATES 

SARs  have  not  always  provided  a  full  explanation  for 
changes  to  a  system's  development  estimate.  In  addition, 
explanations  for  changes  are  normally  provided  on  a 
'one-time"  basis,  and  subsequent  SARs  do  not  note  that 
the  development  estimate  was  revised.  Of  51  major  systems 
included  in  the  reporting  system  as  of  June  30,  1979, 

19  changed  their  development  estimates  for  performance, 
schedule,  or  cost  34  times  since  1970.  Of  the  changes, 

12  were  to  provide  additional  performance  specifications 
or  schedule  milestones.  The  remaining  22  changes  were 
revisions  to  previously  reported  data. 


The  significance  of  the  changes  could  not  always  be 
determined  from  the  explanation?*  provided.  For  example,  the 
Harpoon  and  DDG-47  SARs  reported  changes  in  missile 
dimensions  and  in  displacement,  respectively,  but  contained 
no  explanation  of  the  impact,  if  any,  of  the  revisions. 
Similarly,  performance  specifications  were  changed  on  two 
occasions  for  the  F-16  without  an  explanation  of  the  impact 
of  the  changes.  The  combat  radius,  ferry  range,  and  engine 
thrust  values  were  reduced,  and  the  thrust-to-weight  ratio 
and  acceleration  value  were  increased.  The  explanations 
provided  were  that  the  development  estimates  were  revised 
to  (1)  agree  with  F-15  data  for  the  F-100  engine  and  (2) 
reflect  a  "for  coordination"  DCP. 

For  another  system,  the  Light  Airborne  Multi-Purpose 
System  MK  III,  the  scheduled  initial  operational  capability 
date  was  extended  by  15  months.  The  revision  was  attributed 
to  "full-scale  development." 

As  another  example,  the  December  31,  1977,  SAR  on 
the  F-18  reduced  the  development  estimate  values  for  speed, 
combat  ceiling,  and  rate  of  climb  and  increased  the 
estimated  weight.  According  to  the  explanation  provided, 
the  changes  were  made  to  represent  the  project  manager’s 
best  estimate  based  on  completion  of  the  final  design 
review.  This  change,  in  our  opinion,  limits  the 
visibility  and  trackability  provided  by  subsequent  SARs. 

We  also  found  that  only  eight  SARs  which  immediately 
succeeded  those  reporting  development  estimate  changes 
referred  to  the  changes.  Only  three  June  30,  1979,  SARs 
indicated  that  their  development  estimates  had  been 
changed . 

Concern  has  long  existed  about  the  need  to  provide 
complete  visibility  and  trackability  of  a  program's  pro¬ 
gress.  For  example,  in  August  1974  the  Senate  Appropriations 
Committee  reported  (Report  No.  93-1104)  that 

"Changes  in  planning  and  development  estimates 
should  not  be  deleted  from  subsequent  reports. 

SARs  should  contain  a  cumulative  record  of  all 
estimates  so  that  there  is  total  visibility  and 
trackability  from  a  program's  inception." 

Moreover,  in  testimony  before  the  House  Subcommittee  on 
Legislation  and  National  Security,  Committee  on  Government 
Operations,  in  June  1979,  the  Deputy  Secretary  of  Defense 
noted  that  the  SAR  focuses  primarily  on  exceptions  to  the 
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approved  baseline  plan  and  includes  the  explanation  for 
all  variances  from  the  program  baselines. 

REPORTING  OPERATIONAL  AND  TECHNICAL  RISKS 

Neither  the  SAR,  nor  the  budget  justification  data,  nor 
prepared  statements  for  budget  hearings  fully  present  the 
operational  and  technical  risks  associated  with  weapon  system 
development  and  planned  use.  On  occasion  these  risks  are  dis¬ 
cussed  during  hearings  in  answer  to  specific  questions.  Pre¬ 
sentation  of  data  to  the  Congress  in  this  manner  may  occur 
long  after  the  data  is  known  and  has  been  considered  by  DOD. 
DCPs  document  Defense  Systems  Acquisition  Review  Council 
concerns  and  recommendations  and  often  contain  data  on 
operational  and  technical  risks  in  either  a  section  on 
operational  and  technical  risk  or  a  section  on  critical 
test  issues. 

For  example,  the  Hellfire  and  Copperhead  DCPs  identify 
operational  and  technical  risks  and  critical  test  issues  which 
cover  many  of  the  environmental  and  operational  factors  which 
will  degrade  system  performance  such  as  weather,  counter- 
measures,  and  battlefield  conditions.  As  another  example, 
the  Stand-Off  Target  Acquisition  System  DCP  lists  aircraft 
survivability  as  a  program  risk. 

In  discussing  the  possible  addition  of  such  a  section  to 
the  SAR,  some  DOD  officials  either  questioned  whether  the 
Congress  needs  or  wants  such  information  or  stated  that  such 
data  is  now  provided  on  request.  In  our  opinion,  this  data 
is  essential  to  making  informed  decisions  on  major  programs. 

It  would  alert  the  Congress  earlier  of  potential  system  limi¬ 
tations  and  would  provide  information  which  would  put  the  of¬ 
ten  subjective  and  seemingly  optimistic  current  estimate  of 
system  performance  into  better  perspective.  Moreover,  formal 
routine  presentation  of  such  data  would  be  a  more  effective 
way  to  provide  the  information  to  the  Congress. 

i 

ADDING  AND  DELETING  SYSTC.nS 

DOD  accepted  our  March  1975  recommendation  to  provide 
precise  criteria  for  adding  and  deleting  systems  from  SARs 
and  revised  the  SAR  Instruction  which  currently  states: 

"SAR  Add  it,,  ns.  New  SARs  will  normally  be  limited  to 
tho  ,e  majoi  systems  which  have  received  Milestone  II 
approval.  The  addition  of  a  new  SAR  will  be  automa¬ 
tic  with  approval  for  the  system  to  enter  full-scale 
engineering  development,  i.e.  Milestone  II  approval  or 
release  of  Engineering  Development  (6.4)  funds." 
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"SAR  Deletions,  Termination  of  SAR  reporting  will 
normally  be  considered  when  production  of  the  system 
is  90  percent  complete  (i.e.,  either  90  percent  of 
expected  deliveries  have  been  made  or  90  percent  of 
expected  outlays  have  been  expended)  and  the  pro¬ 
gram  is  no  longer  a  P-1  line  item.  Termination  of 
SAR  reporting  is  not  automatic  and  must  be  request¬ 
ed  and  approved  prior  to  the  'as  of*  date  of  the 
last  proposed  submission.  *  *  ** 

The  instruction  also  calls  for  earlier  addition  to  or  deletion 
from  SAR  coverage  based  on  recommendations  from  dod  components 
or  appropriate  congressional  committees. 

This  revision  is  an  improvement.  In  our  opinion,  how¬ 
ever,  not  enough  consideration  is  being  given  to  adding  impor¬ 
tant  systems  not  yet  in  full-scale  engineering  development  to 
the  reporting.  Early  and  adequate  visibility  over  new  weapon 
systems  is  necessary  if  informed  decisions  are  to  be  made. 

The  SAR  system  could  help  provide  this  visibility.  To  demon¬ 
strate  the  need  for  earlier  SAR  reporting,  the  following 
chart  shows  systems  recently  added  to  the  SAR,  the  funds 


appropriated  by 
initial  funding 

the  time 
for  the 

Fiscal 

year 

of  their 
system . 

first  SAR,  and 

Funding  at 

the  year  of 

f  irst 

Date  of 

time  of 

Development 

System 

funded 

initial  SAR  initial  SAR  estimate 

- (millions) - 

Multiple  Launch 

Rocket  System 

a/1976 

6-30-79 

$125.5 

b/$3 ,453.8 

Pershing  11 

1975 

3-31-79 

152.9 

1,571.0 

HARM 

Stand-Off  Target 
Acquisition 

1972 

9-30-78 

123.9 

c/  2,409.9 

System 

a/1974 

9-30-78 

37.6 

1,282.8 

PLSS 

1972 

3-31-78 

57.0 

954.5 

Tomahawk 

Air-Launched 

1973 

12-31-77 

502.4 

2,422.9 

Cruise  Missile 

1975 

12-31-77 

582.7 

4,181.9 

a/ Approximate  date. 
b/Planning  estimate. 

c/Excludes  Air  Force  funding  of  $6.1  million  in  1977. 
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As  shown  above,  the  current  criteria  for  adding  weapon 
-/stems  to  SARs  permits  them  to  be  in  development  for  extended 
periods  of  time  and  to  have  large  sums  of  money  expended  on 
them  before  SAR  coverage.  Many  major  weapon  systems  are 
currently  excluded  from  SARs  because  they  have  not  been  ap¬ 
proved  for  full-scale  development.  Current  systems  in  this 
category  include  the  joint  Tactical  information  Distribution 
System, *Navstar ,  Sound  Surveillance  System  Improvement, 
Advanced  Anti-Submarine  Warfare  Torpedo,  MK-46  Torpedo 
Near-Term  improvement,  and  Single  Channel  Ground  and 
Airborne  Radio  Subsystem. 

In  our  opinion,  there  should  also  be  greater  considera¬ 
tion  to  deleting  systems  from  SAR  coverage  earlier  than  the 
normal  criteria — 90  percent  completion  of  deliveries  or 
expenditures.  We  believe  it  is  not  as  critical  to  have 
SAR  coverage  when  a  system  nears  completion  of  production 
as  in  earlier  stages  of  the  acquisition  process  when  more 
critical  decisions  are  made.  Current  systems  that  we  believe 
should  be  considered  for  deletion  from  SAR  coverage  are  the 
A- 1 0 ,  Airborne  Warning  and  Control  System,  F-15,  Harpoon, 
and  P-3C. 

INCLUDING  LOGISTIC  SUPPORT 

AND  ADDITIONAL  PROCUREMENT  COSTS 

Logistic  support  and  additional  procurement  costs  amount 
to  hundreds  of  millions  of  dollars  on  some  systems  and  can  be 
a  major  factor  in  deciding  whether  to  buy  a  weapon  system. 
Examples  of  these  costs  include  replenishment  spares,  modifi¬ 
cations,  component  improvements,  common  support  equipment, 
production  base  support/facilities,  and  other  costs  listed 
in  the  Five-Year  Defense  Plan  procurement  Annex. 

As  weapon  system  costs  increase,  these  types  of  costs 
will  have  even  greater  impact  on  management  decisions. 

Since  its  September  23,  1975,  revision,  DOD's  SAR  instruction 
has  not  required  reporting  of  logistic  support  and  additional 
procurement  costs.  In  our  opinion,  these  costs  should  be 
included  in  the  cost  section,  and  changes  from  one  period 
to  another  should  be  explained. 

As  the  SAR  process  evolved,  DOD  made  frequent  changes 
in  this  area.  The  SAR  instruction  was  revised  on  June  12, 
*970,  to  require  that  additional  procurement  costs  (subse¬ 
quently  called  logistic  support/additional  procurement 
costs)  be  included.  These  costs  were  defined  as  all  pro¬ 
curement  costs  related  to  maintaining,  operating,  or  improving 
a  major  defense  system  in  addition  to  its  program  acquisition 


In  a  letter  dated  May  25,  1972,  the  Assistant  Secretary 
of  Defense  (Comptroller)  issued  new  reporting  requirements 
for  logistic  support/additional  procurement  costs.  The  letter 
stated  that,  in  the  interest  of  uniformity,  clarity,  and 
simplicity  of  the  reporting  requirement,  only  modification 
and  component  improvement  costs  would  be  reported.  All  other 
costs  being  reported  were  dropped.  In  our  March  1975  report 
on  the  SAR  system,  we  suggested  the  followings 

"Considerable  improvement  could  be  made  in  report¬ 
ing  of  logistic  support/additional  procurement 
costs.  They  should  be  expanded  to  include  all 
remaining  procurement  costs  related  to  a  program 
but  not  currently  being  reported  as  program  acqui¬ 
sition  costs.  They  should  also  be  included  in 
the  cost  section  of  the  SAR  rather  than  being 
reported  in  a  separate  section,  in  addition, 
this  section  on  logistic  support/additional  pro¬ 
curement  costs  should  include  firm  baselines 
established  with  footnotes  indicating  the  basis 
for  these  baselines,  and  any  changes  from  these 
baselines  should  be  provided  in  the  form  of  a 
variance  analysis." 

As  stated  above,  in  September  1975  the  revised  SAR  in¬ 
struction  deleted  the  requirement  for  reporting  logistic 
support  and  additional  procurement  costs. 

To  illustrate  the  extent  of  funding  not  included  in 
selected  acquisition  reporting,  we  requested  that  the  Office 
of  the  Secretary  of  Defense  provide  cost  estimates  for 
logistics  support/additional  procurement  costs  for  a 
limited  number  of  systems.  The  following  chart  shows 
the  cost  estimates  provided  for  the  period  fiscal  year 
.1978  through  1984  for  six  systems  and  their  program 
acquisition  cost  estimates.  The  chart  shows,  for  example, 
that  estimated  logistics  support/additional  procurement  costs 
for  the  P-3C  for  fiscal  years  1978  through  1984  are  $709  mil¬ 
lion.  The  program  acquisition  cost  for  this  program  is 
$5,485  million.  We  did  not  review  these  cost  estimates 
to  determine  their  accuracy  or  completeness. 
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Fiscal  year  1978  through  1984  estimates  of 
logistic  support/additional  procurement 


System 

Mod ifi— 
cation 

Compo¬ 

nent 

improve 

ment 

Replen¬ 
ishment 
spares 
(note  a) 

Other 
(note  b) 

Total 
(note  c) 

Program 
acquisi¬ 
tion  cos 

P-3C 

$658 

$  6 

$  44 

$  709 

$  5,485 

F-14A 

522 

62 

- 

13 

596 

12,191 

A-l  0 

294 

118 

- 

93 

505 

4,812 

E-  3  A 

181 

9 

- 

- 

190 

4,147 

XMl 

(d) 

(d) 

- 

340 

340 

10,926 

F-l  6 

758 

200 

- 

182 

1,140 

15,051 

a/DOD  officials  were  not  able  to  identify  replenishment  spares 
cost  estimates  for  the  individual  systems. 

D/Includes  (1)  industrial  facilities/production  base,  (2) 
simulators  (other  charges),  (3)  consumables,  and  (4)  modi¬ 
fication  spares. 

c/Totals  may  not  add  due  to  rounding. 

d/As  of  the  President's  fiscal  year  1980  budget,  no  procure¬ 
ment  improvement  program  had  been  established  for  the 
XM1  tank  program  and  no  wartime  life-cycle  cost  effort 
had  been  made. 

In  1977  the  Congress  directed  DOD  to  include  in  the 
F-15  SAR  the  component  improvement  program  funds  related 
to  the  aircraft's  engine.  Also  the  Navy  portion  of  the 
HARM  SAR  reports  over  $100  million  for  command  and  launch 
equipment.  These  costs  are  not  shown  as  part  of  the  program 
cost  estimate  but  are  listed  as  additional  costs. 

These  examples  illustrate  the  desire  on  the  part  of  the 
Congress  to  have  these  types  of  costs  reported  and  the  incon¬ 
sistency  of  reporting  between  SARs  that  report  part  of  these 
costs  and  SARs  that  do  not.  We  believe  DOD  should  report 
logistic  support/additional  procurement  costs  on  the  SAR. 

The  DOD  position  has  been  that  the  logistic  support/ 
additional  procurement  costs  are  unrelated  to  acquisition 
costs,  are  not  under  the  direct  control  of  the  project  mana¬ 
ger,  and  should  not  be  reported  on  the  SAR.  The  DOD  response 
to  our  March  1975  report  advised  that,  as  the  capability  to 
estimate  life-cycle  costs  improves,  cons  ide rat  ion  may 
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he  given  to  including  these  estimates  in  the  SAR.  In  addi¬ 
tion,  DOD  officials  said  that  draft  revisions  to  SAR 
instruction  were  forwarded  to  oversight  committees  in 
1975  and  1979  and  comments  were  requested.  However, 
no  comments  were  received. 

PROVIDING  COST  ESTIMATES  BASED  ON 
VARIOUS  ESCALATION  RATES 

Before  June  30,  1979,  DOD  included  a  chart  in  the  SARs 
showing  how  much  the  total  program  acquisition  cost  would 
decrease  or  increase  if  various  escalation  rates  were  used. 
These  rates  were  different  from  the  rate(s)  used  in  devel¬ 
oping  the  SAR' s  acquisition  cost  estimate.  For  example, 
the  March  31,  1979,  SAR  for  the  F-18  included  a  total  pro¬ 
gram  acquisition  cost  estimate  of  over  $24  billion,  including 
$11.5  billion  for  escalation  based  on  a  5.6-percent  annual 
rate.  The  chart  described  above  showed  that  the  total 
program  acquisition  cost  would  vary  if  different  inflation 
factors  were  used,  as  follows: 


Escalation  rate 


Resulting  change  to 
program  cost  estimate 


( percent) 


(millions) 


2 

4 

6 

8 


$-5,370 

-2,758 

622 

4,580 


DOD  officials  stated  that  this  information  is  not  needed. 
Their  opinion  was  based  on  (1)  no  indication  from  their 
oversight  committees  that  the  data  was  needed  when  the  latest 
revision  to  the  SAR  instruction  was  submitted  for  comment 
and  (2)  no  request  from  their  oversight  committees  to  include 
this  data  after  it  was  omitted  from  recent  SAR  submissions. 

In  our  opinion,  since  the  escalation  rates  used  in  SARs  have 
traditionally  been  lower  than  actual  rates,  such  a  chart 
served  a  useful  purpose  We  believe  that  DOD  should  reinsti¬ 
tute  the  chart  and  should  include  escalation  rates  at  least 
as  high  as  the  approximate  rate  being  experienced  when  the 
SARs  are  being  prepared, 

CONDUCTING  PERIODIC  INDEPENDENT  REVIEWS 


In  out  opinion,  one  of  the  most  important  SAR  improve¬ 
ments  needed  is  a  periodic  independent  review  of  the  SAR 
system  by  the  Office  of  the  Secretary  of  Defense  to  insure 
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that  SARs  are  providing  consistent  and  reliable  data  and 
full  disclosure  of  the  status  of  major  weapons  systems. 

During  the  years  the  SAR  has  been  in  existence,  DOD's 
SAR  review  efforts  have  been  directed  primarily  toward 
evaluating  recommendations  to  improve  SAR  format  and  guide¬ 
lines.  During  1974  a  review  was  made  in  response  to  recom¬ 
mendations  by  the  Congress,  elements  of  DOD,  and  us.  As  a 
result,  a  revised  SAR  instruction  was  issued  on  September  23, 
1975.  In  1978  the  SAR  instruction  was  reviewed  again,  and 
changes  were  made  in  format  and  data  requirements.  A  revised 
SAR  instruction  was  issued  on  April  4,  1979.  While  the  SAR 
instructions  and  procedures  have  been  improved,  DOD  has  not 
made  an  audit,  review,  or  evaluation  to  assure  itself  that 
the  instructions  are  being  properly  implemented. 

DOD  has  taken  the  position  that  since  the  responsibility 
for  preparing  SARs  has  been  delegated  to  weapon  system 
program  managers,  they  are  accountable  for  their  accuracy 
and  completeness.  There  are  SARs  on  about  50  weapon  systems 
at  any  time,  and  program  managers  have  interpreted  the 
instructions  in  different  ways.  Another  concern  we  have 
results  from  the  fact  that  SARs  provide  a  means  of  assessing 
how  successfully  a  program  manager  is  accomplishing  program 
goals.  Accordingly,  the  program  manager,  as  the  program  ad¬ 
vocate,  is  optimistic  about  the  program.  It  is  essential  that 
program  managers  be  optimistic  advocates  of  their  systems, 
but  it  is  also  essential  that  the  Congress  be  provided  status 
information  that  is  as  complete,  accurate,  and  objective  as 
possible.  We  believe  that  periodic  independent  reviews  of 
the  SAR  system  would  aid  in  providing  this  kind  of  data. 

Examples  noted  during  our  review  of  conflicting,  incom¬ 
plete,  or  misleading  data  reported  in  SARs  include 

—  incomplete  test  results  reported  on  the  HARM  SAR  re¬ 
lated  to  fuze  performance  and  on  the  Roland  SAR  related 
to  tracking  tests; 

— misleading  or  incomplete  data  in  the  operational  and 
technical  characteristics  section  of  the  PLSS  (see 
p.  8),  Copperhead  (see  p.  7),  and  F-18  1/  SARs;  and 


1/Although  the  Navy  has  carried  out  flight  tests  to  evaluate 
F-18  performance,  including  acceleration,  range,  and  main¬ 
tainability,  it  does  not  plan  to  report  demonstrated  per¬ 
formance  data  until  development  aircraft  number  9  is  flown 
in  .Tune  1980. 
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— misleading  cost  data  in  the  Captoc  SAR  because  it 
does  not  identify  substantial  costs  for  torpedoes — 
part  of  the  weapon  system — which  will  be  taken 
from  Navy  inventory  but  which  will  subsequently 
be  replaced  with  new  purchases. 

CERTIFICATION  OF  SARs 


In  our  previous  report  we  recommended  that  project 
officers  be  responsible  for  the  completeness,  reasonableness, 
and  accuracy  of  SARs  and  that  they  certify  as  to  the  credi¬ 
bility  of  the  reports.  Our  recommendation  was  based  on  our 
concern  that  the  many  review  levels  to  which  SARs  are  sub¬ 
jected  result  in  changes  and  additions  without  full  coordi¬ 
nation  with  the  project  officer.  DOD  objected  to  our 
recommendation  because,  based  on  DOD's  interpretation,  it 
would  have  denied  the  military  service  Secretaries  and 
the  Secretary  of  Defense  their  responsibility  for  reviewing 
SARs  before  submission  to  the  Congress. 

Our  concern  still  exists,  especially  in  view  of  the 
problems  identified  in  our  report.  Since  the  SAR  is  issued 
from  the  Office  of  the  Secretary  of  Defense  and  some  of 
our  current  recommendations  would  expand  the  scope  of  the 
SAR,  the  Secretary  of  Defense  or  his  designee  should,  in 
our  opinion,  certify  to  the  credibility  of  the  reports. 

RECOMMENDATIONS 

We  recommend  that  the  Secretary  of  Defense  revise  the 
SAR  instruction,  where  necessary,  and  enforce  the  instruction 
so  that  SARs  include 

— a  mission  capability  assessment  statement,  including 
expected  shortcomings  and  limitations  of  the  system 
in  its  operational  environment; 

— status  of  key  subsystems  and  related  systems, 
including  related  systems  on  separate  SARs; 

— planning  estimates  with  a  one-time  explanation  for 
changes  to  arrive  at  the  development  estimates; 

— ranges  of  costs  for  the  planning  and  development 
cost  estimates  rather  than  specific  point  estimates; 

— more  complete  explanations  for  changes  to  devel¬ 
opment  estimates  and,  in  subsequent  SARs,  a 
reference  to  the  original  development  estimates; 
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— a  section  on  operational  and  technical  risks; 

— logistic  support/additional  procurement  costs  and 
explanations  for  changes; 

— a  chart  showing  the  impact  on  the  program 
acquisition  cost  estimate  of  using  different 
escalation  rates;  and 

— a  certification  to  the  credibility  of  SARs  by  the 
Secretary  of  Defense  or  his  designee. 

In  addition,  we  recommend  that  the  Secretary  of 
Defense  direct  that  a  periodic  review  be  made  of  the  accuracy 
and  completeness  of  SARs  and  that  greater  consideration  be 
given  to  (1)  adding  important  systems  in  advanced  development 
to  the  reporting  system  and  (2)  deleting  older  systems  from 
the  reporting. 

We  recognize  that  it  is  difficult  for  DOD  to  decide 
what  information  to  include  in  the  SARs  and  in  what  detail. 

DOD  must  make-  the  SARs  short  enough  to  be  usable  by  people 
who  have  little  time  to  review  them,  and  yet  SARs  should 
present  data  that  is  complete,  accurate,  and  not  misleading. 
Also,  the  additional  data  we  are  recommending  for  inclusion 
(such  as  operational  and  technical  risks,  operational  capa¬ 
bility  shortfalls,  additional  costs  needed  to  deploy  a  system, 
and  planning  estimates)  is  the  kind  of  data  DOD  may  not  want 
to  include  because  it  detracts  from  an  optimistic  presentation 
of  system  capabilities  and  program  progress  and  status.  How¬ 
ever,  it  seems  to  be  the  kind  of  data  the  Congress  needs  to 
have  in  reviewing  and  funding  programs. 


(951487) 
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